NOVA - For the NeXT Workstation/NOVA - For the NeXT Workstation.iso/Newsletters/GEnieUnixNews/unxnl-06.91

Text File | 1992-12-27 | 20KB | 485 lines

          _ __  _   _ __ _ __
         // /  //| // || \\|     N E W S
        //_/  // |//  ||  |\\    Vol 2, Issue 3 - June 1991
         R o u n d T a b l e

   Items of interest to participants of the GEnie Unix RoundTable

      The RoundTable SysOps are:
      Dave Weinstein......OLORIN     Brian Riley.........DELPHI
      Gary Smith..........GARS       Chris North-Keys....HARP
      Rick Mobley.........LRARK    All Unix SysOps.....UNIXSYSOPS$
   We strongly encourage you to contact any or all of us if you have -ANY-
 comments or suggestions. This is -YOUR- RoundTable. We are here to make
 your participation as pleasant and beneficial as possible.

 ED: editor notes - AWK rules (this issue)
 --
   In recent issues I have included a few notes on AWK usage and some example
 scripts. I was both surprised and pleased that many were quite interested in
 AWK and wanted more of the same. With that in mind I endeavored to collect
 several worthwhile examples from news articles. The end result is this will
 be the second newsletter that supports a central theme - the theme this issue
 is obviously AWK scripts.
   Would more of the same be in order for a future issue, or would readers be
 interested in sed or perl ? Please leave comments or votes via GEnie e-mail
 to GARS.

 GUI Article - continued
 -----------
     by Dave Weinstein (OLORIN)

    [ Ed. note: Dave began this look at GUI's in Unix RoundTable newsletter
                volume 2, issue 2 available in GEnie Unix RoundTable library
                1 as UNXNL-04.91 ]

       The continuing GUI Saga

   It's done. It's in Beta. It's not working *quite* the way you expected.

   Macro recorder's are your friend. Requiring all bug reports to accompany
 a recording of all of the various movements, clicks, and keystrokes required
 to break your code can make your life easier (especially if your duties
 involve bug xifixes and support, continuing upgrades to the Beta version,
 and preliminary work on the next set of features to be added) [Note: any
 similarities between the described conditions and the author's current state
 are due to the fact that he has spent too much time fixing bugs and not
 nearly enough time doing anything else]. Many times, the user's don't think
 right (or at least, they don't think the way you do), and watching exactly
 what they do can be the difference between quickly finding and fixing a
 problem and spending 30 hours trying to figure out just what the users mean
 by "it doesn't do the window right".

   On the other hand, you may be trying out a new GUI based product. You
 can make the developer's life easier (and get fixes faster), by being
 very clear in describing what has broken, how, and when, and by duplicating
 the problem on your end before calling in the bug.

   Flip through the trade journals; it's getting impossible to turn a
 page without looking at information on everyone's graphics based product.
 More and more mainframe applications are aquiring graphical interfaces.
 Using them, writing them, and figuring out why they don't do what the vendor
 says they do is an easier task if everyone can agree on what was done to
 make them fail.

  --Dave

  FAST and NASTY, DOWN  and  DIRTY:  quick  fix scripts that do something
 --------------------------------
    no quick scripts this issue so we can devote more space to AWK usage.
      Gary

 AWK USAGE - Several news articles collected to demonstrate awk's power
 ---------   and some scripts to demonstrate usage.

 MAKING AWK OUTPUT INTO A VARIBLE :
 ................................

 In article <[e-mail]>,
                      [e-mail] (Thomas Ravnholt) writes:
 |> Hello !
 |>
 |>   I have a little question about awk (nawk).
 |>
 |>   If I want to run a unix-command in an awk-script,
 |> how do I get the output into a variable.
 |>
 |>   I tried
 |>
 |>      getline < system(unixcommand)
 |>
 |>      system(unixcommand | getline)
 |>
 |>
 |> but it is no good of course. system returns 0 or 1 and
 |> not the output of the unixcommand.
 |>

 Crude, but effective, temporary files are your friend.  Here's a
 simple example that does what you ask.

 #! /bin/sh

 awk '
 BEGIN {
 system("date > /tmp/foo")
 getline X < "/tmp/foo"
 print X
 }'

 You can use the pid if you are worried about stepping on duplicate
 temp file names and insert a "trap" command to delete your temp
 file(s) in case your script terminates prematurely.

      -Mark
 --
                            ___  ___
                           / __||__ \
 Mark Kennedy             ( |__  __| )  AT&T: [phone]
 Control Data Corporation  \___||___/   E-Mail: [e-mail]
             check-ins happen

 MAKING CALLS FROM AWK :
 .....................

 From Usenet comp.unix.shell: a Summary on making calls from awk:

 From: [e-mail] (Scott "gaspo" Gasparian)
 Subject: SUMMARY: using sh vars in awk calls.
 Summary: using /bin/sh vars in awk calls

 greetings all!

 Ok, thanks to all who replied, and since I got a couple "me toos",
 heres the deal:

 Problem:
         in a /bin/sh script, need to pass a regular sh variable
         to the awk call, but the proper syntax is not-readily
         discernable.

 Example:

         given sh vars $fred, call awk using fred, like so:

         wilma=`cat $filea | awk '{if($1==fred){print("match")}}'`

 Solution:
         (1), escape out the sh variable by using single quotes.

         [e-mail] (Robert Hartman) says:
 >
 >        .... awk '{ ... '$fred' ...}' ...
 >
 >Since single-quotes suppress variable substitution, if you want the shell
 >to expand the reference you have to leave it outside.  Note that
 >the whole awk argument is still treated as one word by the shell, since
 >there is no white space between the single quotes and the expanded shell
 >variable.

         Kenneth Herron <[e-mail]> agrees with a slightly
         different syntax of
 >
 >         wilma=`cat ${file1} | awk '{if($1== "'$fred'"){print("match")}}'
 >       or
 >         wilma=`cat ${file1} | awk "{if(\$1==\"$fred\"){print(\"match\")}}"

         (2), the other possible way is to declare awk a variable after
         the end of the awk script (in the man pages, but I cant find it)
         like so:

         [e-mail] (David Neill-OKCy Mktg 405-278-4007) suggests

 >               wilma=`cat $file1 | awk '
 >               $1 == fredvar {printf("match on %s\n",fredvar)}
 >               ' fredvar="$fred"`

          and [e-mail] (Andy Behrens) explains:

 >    fred="barney"
 >    wilma=` cat ${file1} |
 >          awk '{
 >                   if ($1 == fred {print("match")}
 >                }' fred=${fred} - `
 >
 >Awk allows command line assignments of the form 'name=value'.  These
 >assignments follow the awk script and precede the filenames.  In most
 >versions of awk, you must have at least one filename on the command
 >line.  If you want awk to read from standard input (as in the above
 >example), you can use a filename of '-'.

 I've only tried the single quote method, and it works fine.  Some of
 this is supposed to be dependent on your version of awk.  dont ask,
 i dont know.

 Thanks for all the info!

 --gaspo.

 BTW, to the dude who thought my sig was "long and obnoxious", you wouldnt
 like my hair either!

 /----------------------------------------------------------------------------\
 | Scott "gaspo" Gasparian  --  System Administrator |"Do you go every Sunday?|
 | Dept. Informatik, Eidg. Techn. Hochschule, Zurich | sounds to me like YOU  |
 | ETH-Zentrum,  CH-8092  Zurich.  T# 01-01-254-7205 | are the addicted one." |
 | [e-mail] | "Good friends we've had, or good friends we've lost,  |
 | ..!ethz-inf!gaspar | along the way.In this proud land,you can't forget your|
 | [e-mail] | past,so dry your tears I say. No woman, No cry." -BMW |
 \----------------------------------------------------------------------------/

 AWK SCRIPT TO DELETE EMPTY LINES :
 ................................

 Kristopher Stephens and and Mike Moore respond to a news request
 for help generating a script to delete empty lines in a text file.

  First, from Kristopher Stephens:

 How about this:

         :
         # sh or ksh
         #
         # Reduce all multiple consecutive "empty" lines to singles.  Process
         # stdin (switch it to filename processing or either as you wish).
         #

         # Make the T in the sed line a <TAB> char.  I leave it a T for
         # this posting just for clarity.

         # Use sed to strip all trailing blanks and tabs
         sed 's/[ T]*$//' |

         # Use awk to delete empty lines that follow empty lines
         awk '
                 $0 != "" {
                         print
                         prevnull = 0
                         next
                 }

                 prevnull == 0 {
                         print
                         prevnull = 1
                 }' -

 The flow in the awk script is to look for non-null lines first on the
 assumption that more input lines will be non-null than null.  If a line
 is non-null, print it and set the null-marker off.  No need to test the
 marker variable before setting it, that would take longer than simply
 making the assignment every time.

 If we get past that first block, we are processing a null line.  If the
 null-marker is off, print this null line and set the null-marker on.  Any
 line not matching one of these two blocks is a null line following a null
 line, and is ignored.

 This is a pretty standard approach for me to take.  I use sed to do editing
 of an input-stream for awk, where I do the conditional work based on line
 contents.  Since sed is faster than awk, the sed process typically writes
 the data out faster than awk will take it in, so doing the line-editing in
 sed instead of within the awk script is no performance penalty at all for
 the shell script as a whole.  It is, in fact, a performance gain because
 the awk script would take more time if I expanded its definition of a null
 line to include blanks and tabs.

 ...Kris
 --
 Kristopher Stephens, | (408-746-6047) | [e-mail] | KC6DFS
 Amdahl Corporation   |                |                    |
      [The opinions expressed above are mine, solely, and do not    ]
      [necessarily reflect the opinions or policies of Amdahl Corp. ]


 ...and from Mike Moore:

 The sed script below works, but messes up on lines at the beginning
 and end of the file.  It may also have problems with large files.
 The awk script works wonderfully!

 in=`cat $1 | tr '\012' '^L'`

 echo $in | sed -e 's/^L^L[^L]*/\
 \
 /g' -e 's/^L/\
 /g'

 #========================

 awk ' BEGIN { blank=0
               line=0
             }
             {
 # remove here...
               if ( blank == 0 )
               {
                 if ( $0 != "" )
                 {
                   blank++

                   if ( line != 0 )
                     print ""

                   print $0
                 }
                 else
                   line++
               }
               else
               {
 # to here, if you do not want to account for blank lines at beginning

                 if ( $0 == "" )
                   line=1
                 else
                 {
                   if ( line != 0 )
                   {
                     print ""
                     line=0
                   }

                   print $0
                 }

 # (and this...)
               }

             }

 # remove here...
       END   { if ( line != 0 )
                 print ""
             }
 # to here, if you do not want to account for blank lines at end

             ' $1


 #========================

 --
 ---            | usual and obvious disclaimers etc...      | Never take a
 Mike Moore     | (anyone daft enough to sue me deserves    | Scorpio seriously.
 [e-mail]   |  to win a share in my negative cashflow!) | I know, I am one.

 DIAL UP COMMUNICATIONS TUTORIAL : - Understanding modem terminology
 -------------------------------
                              Modem Talk
                           by Ricky Mobley

 Modems come in many shapes and sizes and from many manufacturers.
 Some manufacturers produce several styles themselves with many
 different features to encompass a wide variety of users. It is very
 interesting, as well as a good exercise, to compare the specifications
 of these modems and to purchase the model that will best serve you.

 Most people are very satisfied with purchasing an asyncronous modem,
 which sends each byte independently. Using this method ensures
 compatibility with a large group of existing modems, but is slower and
 sometimes more restrictive when it comes to actual data throughput.
 Each byte consists of a start bit plus eight data bits plus one stop
 bit. Thus, we are sending additional bits of information that actually
 contain no useful data.

 Some standards had to be set in order to determine just which modem
 would talk to another modem at the other end of a dial-up line. In
 steps the CCITT, A French acronym for the International Telegraph and
 Telephone Consultative Committee. Their standards, or 'recommendations'
 as they have it, are published every so often in a set of 'fascicles',
 whose color varies with the year. The current set is the 'blue books'
 published in 1988. Listed below are some of these standards and a
 description as to how they apply to our current technology.

 V.21:     300bps. Also known as Bell 103.
 V.22:     1200bps, with fall back to 600bps. Also known as Bell 212.
 V.22 bis: 2400bps, with fall back to V.22.
 V.23:     1200bps with 75bps back channel, with fall back to
           600bps/75bps.
 V.29:     9600bps half duplex or four wire (used by FAX) with fall back
           to 7200bps and 4800bps.
 V.32:     9600bps with fall back to 4800bps.
 V.32 bis: 14400bps with fall back to 12000bps, 9600bps, 7200bps and
           4800 bps.

  Other standards you may encounter:

 V.24:     Connection between DCE and DTE. Effectively the same as
           RS232, though V.24 only specifies the meaning of the
           signals, not the connector nor the voltages used.
 V.25 bis: A cryptic command language for modems.
 V.42:     Error correction with asynchronous to synchronous conversion.
 V.42 bis: Data compression using a Lempel-Ziv related technique, which
           detects frequently occurring character strings and replaces
           them with tokens. This is similar to Unix compress. Typical
           compression for text is 50% or better; with nearly 20% gain
           from synchronous conversion this reduces transmission time
           by almost 60%.

 As you have probably noticed, when you exceed 2400 baud you are
 running half duplex or some form of compression. These techniques are
 further explained as a set of modem-to-modem protocols that provide
 both error correction and compression known as Microcom Network
 Protocols or MNP for short. The commonly encountered ones are:

 MNP2. Error correction using asynchronous transmission.
 MNP3. Error correction using synchronous transmission between the
       modems (the DTE interface is still asynchronous). Since each
       eight-bit byte takes eight rather than ten bits to transmit
       there is scope for a 20% increase in throughput. Unfortunately
       the MNP3 protocol overhead is rather high so this increase is
       not realized.
 MNP4. Introduces `data phase optimization', which improves on the rather
       inefficient protocol design of MNP2 and MNP3. Synchronous MNP4
       comes closer to achieving the 20% throughput increase mentioned
       above.
 MNP5. Simple data compression. Dynamically arranges for commonly
       occurring characters to be transmitted with fewer bits than rare
       characters. It takes account of changing character frequencies
       as it runs. Also encodes long runs of the same character specially.
       Typically compresses text by 35%; with 20% for MNP4 this reduces
       data by almost 50%.

 Many modern modems incorporate MNP5 or V.42 bis to compress data
 before it is sent over the phone line. For this to be effective, the
 modem must be fed data at a higher speed than the phone line speed.
 Compression is most useful for interactive sessions in the modem. If
 you are sending files, compressing them on the computer before sending
 is usually more effective. In this case, make sure that the modem is
 not also trying to compress because already-compressed data will
 usually become larger if re-compressed.

 For compression to work, it is essential that the data is sent over
 an error-free link. Otherwise the modems could get out of sync and
 hopelessly garble the data. Since common error correction protocols
 are synchronous, there is usually a throughput gain there as well.

 Manufacturer claims that MNP5 provides a 2:1 reduction in size, or
 that V.42 bis provides 4:1, should be taken with a grain of salt.
 These claims are only true for suitable data.

 Like compression, error correction is most useful for interactive use.
 When sending files, it is generally best to let the computers at each end
 do the correction, using a protocol like Kermit or uucp. However, the
 ability of MNP4 and V.42 to send data synchronously may make it
 worthwhile to use them.

 A discussion about modulation technique would not be complete without
 mentioning PEP (Packetized Ensemble Protocol) devised by Telebit and
 used in their Trailblazer modems. It can achieve throughput of 9600bps
 or better, and is reported to be more resilient than V.32. However, it
 is half duplex with a long turnaround time. So for file transfer it
 generally has to be used with protocol spoofing.

 I have just touched on this subject and as you can see, it takes a lot
 to get good reliable data from one end to another. There are many
 methods to do it and lots of good usable equipment on the market today
 to acomplish your task. Hopefully, with this little bit of data you
 will better understand modem talk and be more knowledgable when you
 are searching for that major modem purchase.

 ---------------
    REMINDER - This newsletter is being sent to you 'by request'. If you do
 not wish to keep receiving it, e-mail a stop notice to GARS. On the other
 hand, we would very much appreciate it if you would pass the word that we
 do distribute this item near the tenth (10th) of the month of issue to any-
 one on GEnie who requests it, and will gladly add any name that is requested
 via the same route... e-mail to GARS.
   P L E A S E  also remember contributions are most welcome. Please e-mail
 items and/or suggestions to GARS.

 (EOF)

 Trademark and Copyright notices:
 Unix is a Trademark of  AT&T; GEnie, LiveWire, and RoundTable are Trademarks
 of General Electric Information Services  Company;  Xenix is a Trademark  of
 Microsoft  Corporation;  Microcom, Microcom Network Protocol and MNP are all
 trademarks or servicemarks of Microcom Corporation.

The contents of this newsletter are copyright (c) 1991 and may be copied whole
or in part only if  original  credit is included. The GEnie UNIX RoundTable is
not affiliated with AT&T.
ay be copied whole
or in part only if  origina